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This report presents interim findings from an independent, 
federally funded evaluation of the abstinence education programs authorized 
under the Personal Responsibility and Work Opportunity Reconciliation Act 
(PRWORA) and defined under Title V, Section 510 (b) (2) (A-H) of the Social 
Security Act. The report draws heavily on 4 years of implementation 
experiences in. a selected group of abstinence education programs funded under 
Section 501. The evaluation addresses: the nature and underlying theories of 
the abstinence education programs; the implementation and operational 
experiences of local communities and schools that have received abstinence 
education funding; and the impacts of abstinence education programs on the 
attitudes and intentions of youth to remain abstinent, on their sexual 
activity, and on their risks of pregnancy and sexually transmitted diseases. 
Data from program documents, program observations, focus groups, and 
interviews with participants highlight several lessons learned: Section 510 
abstinence funds are changing the local landscape of approaches to teenage 
pregnancy prevention and youth risk avoidance; most abstinence education 
programs offer more than one message of abstinence; most participants report 
favorable feelings about the program experience; and programs face real 
challenges addressing peer pressure and communication between parents and 
children. An appendix lists the technical workgroup for the evaluation of 
Title V Section 510 Abstinence Education Programs. (Contains 41 references.) 
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Executive Summary 



I n 1996, Congress authorized $50 million annually for five years to promote abstinence 
education. This funding was established through a new formula grant program 
authorized under the Personal Responsibility and Work Opportunity Reconciliation 
Act (PRWORA) of 1996. The funding became available to states in 1998 and is 
administered by the Maternal and Child Health Bureau. States must provide $3 in matching 
funds for every $4 in federal funds, resulting in a total of up to $87.5 million available 
annually for such state programs. Abstinence education programs funded through this new 
grant program teach an unambiguous abstinence message to youth. Programs receiving 
these abstinence education funds may not endorse or promote contraceptive use. 

This report presents interim findings from an independent, federally funded eyaluation 
of the abstinence education programs authorized under PRWORA and defined under Title 
V, Section 510 (b)(2)(A-H) of the Social Security Act. This report draws most heavily on 
four years of implementation experiences in a selected group of abstinence education 
programs funded under Section 510. Later reports from the evaluation will present 
estimates of short- and longer-term program impacts, as well as studies on special topic 
areas. 

Evaluation of Section 510 Abstinence Education Programs 

The evaluation addresses three important questions: (1) What are the nature and 

underlying theories of the abstinence education programs? (2) What are the implementation 
and operational experiences of local communities and schools that have received abstinence 
education funding? and (3) What are the impacts of abstinence education programs on the 
attitudes and intentions of youth to remain abstinent, on their sexual activity, and on their 
risks of pregnancy and sexually transmitted diseases (STDs)? 

The first stage of the evaluation entailed selecting programs for study. The evaluation 
team first visited and observed numerous abstinence education programs across the nation. 
Eleven of these, representing a range of program models and serving different target 
populations, were then selected for in-depth analysis. Five of the 11 are referred to as 
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“targeted” programs: they target services to specific, identifiable groups of youth. The 
remaining six are community-wide, systemic-change initiatives, which use the abstinence 
education funding to increase public awareness of the problems of teen sexual activity, to 
change community norms and attitudes, to encourage stronger parent-child 
communications, and to engage youth in abstinence education and support services. 

The evaluation includes an extensive implementation and process analysis and an impact 
analysis. The implementation and process analysis uses program documents, program 
observations, focus groups with program participants and parents, and interviews with program 
staff and community leaders to document and understand the abstinence education programs 
implemented in the 11 sites. The impact analysis uses longitudinal survey data for groups of 
youth randomly assigned to program and control groups in the 5 targeted program sites. 
Enrollment in the impact evaluation study samples spaimed three school years and was just 
completed in fall 2001. Therefore, the follow-up data that are necessary for the impact analysis 
are not available for inclusion in this report. 

Early Implementation and Operational Lessons 

The first four years of Section 510 funding for abstinence education have generated a 
wealth of experience on how local grantees have designed and implemented abstinence 
education programs. Among the early lessons are the following: 



1. Section 510 abstinence funds are chcuiging the local IcUidscape of 
approaches to teenage pregnancy prevention and youth risk avoidance. 
Despite an initial debate in some states over whether and how to spend 
abstinence education block grant funds, all states applied for funding at some 
point, and most are using monies in innovative ways to promote abstinence 
from sexual activity as the healthiest choice for youth. 

2. Most abstinence education programs offer more than a single message of 
abstinence. Examples of curricula and program components from sites 
participating in the federally funded evaluation indicate the diverse, creative, and 
often complex nature of many initiatives. 



Common Curriculum Topics of Abstinence Programs 
Participating in the Impact Evaluation 



Building Self-Esteem 

Developing Values/Characteir Traits 

Formulating Goals 

Making Decisions 

Avoiding Risky Behavior 

Maximizing Communication 

Strengthening Relationships 

Understanding Development and Anatomy 



Preventing STDs 

Withstanding Social and Peer Pressure 
Addressing Consequences/Self-Control 
Resolving Sexual Conflicts 
Learning Etiquette and Manners 
Aspiring to Marriage 
Understanding Parenthood 
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3. Most participants report favorable feelings about their program 
experience. Youth respond especially positively to staff who show strong and 
unambiguous commitment to the program message. They also like programs 
that deliver an intensive set of youth development services to enhance and 
support the abstinence message. 

4. Abstinence education programs face real challenges addressing peer 
pressure and the communication gulf between parents and children. 
Sexual activity often elicits only casual mention among youth, and is tolerated 
and even promoted by their peer culture. Many programs attempt to address 
peer pressure through parents. Yet, engaging parents has proven to be 
extremely challenging. 

5. Local schools are valuable program partners, but establishing these 
partnerships is sometimes difficult. Their broad access to youth makes 
schools logical and important partners for many programs, but some schools 
resist collaboration with abstinence programs. Sometimes schools resist because 
of competing priorities: at other times, resistance stems from debate about 
health and sex education policies. 

More Lessons to Come 

Congress has authorized a scientifically rigorous, independent evaluation of the 
abstinence education programs funded under Tide V Section 510 to determine the extent to 
which abstinence programs achieve six specific goals: 



1. Strengthen knowledge and attitudes supportive of abstinence 

2. Induce more youth to embrace abstinence from sexual activity as a personal goal 

3. Reduce sexual activity among youth 

4. Persuade sexually experienced youth to become and remain abstinent 

5. Lower the risk of STDs 

6. Lower the risk of nonmarital pregnancies 



Obtaining clear and definitive evidence on the success of abstinence education programs 
in achieving these goals is a difficult task that requires time. Over the past four years, the 
evaluation effort has laid the foundation for a careful, comprehensive, and rigorous impact 
study and has successfully implemented the research design in the five targeted program 
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sites. Study enrollment is completed, and longitudinal tracking of youth through surveys and 
school records is ongoing. 

Critical features of the impact study design now under way are the following: 

The impact evaluation uses an experimental design. In each site, program 
effectiveness will be measured by comparing outcomes of eligible youth who were randomly 
assigned to the program or to a control group. The experimental design offers the only 
means of measuring, with a known degree of certainty, how successful the programs are 
overall and how well they serve key subgroups of youth. Other evaluation designs are 
vulnerable to “selection bias,” which can seriously undermine the credibility of their results. 

The impact evaluation has large sample sizes of between 400 and 700 youth per 
site. Large sample sizes protect against failing to detect true program impacts simply 
because the study lacked statistical power. Three years of study enrollment (fall 1999 
through fall 2001) were necessary to achieve adequate sample sizes. 

The study sample is being followed for up to 36 months. Because so few youth 
engage in sexual activity before entering high school, outcome estimates based on middle 
school youth will miss program impacts on behaviors that could emerge later. The follow- 
up period for the evaluation is such that almost two-thirds of the study sample will be 14 to 
18 years of age by the time of the final survey. 

Analysis and Reporting Plans 

Study enrollment began in fall 1999 and continued through fall 2001. Youth are 
surveyed at or close to study enrollment (wave 1), between 6 and 12 months following 
enrollment (wave 2), and then 18 to 36 months later (wave 3). In sites where program 
participation might be expected to affect school performance, school records will be 
collected through spring 2004. Throughout, data on program operations are being collected 
through observations, executive interviews, program documents, and focus groups. 



Interview Schedule 



Sample Enrollment 



Wave 1 



Wave 2 



Wave 3 



Fall ’99/Spring ’00 



Fall ’99/Spring ’00 



Fall ’00 



Spring/Fall ’02 



Fall ’00 



Fall ’00 



Spring ’01 



Fall ’03 



Fall ’01 



Fall ’01 



Spring ’02 



Fall ’03 
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A report on the effectiveness of the programs in achieving their short-term goals of 
changing knowledge, attitudes, and near-term behavioral choices will be completed in early 
2003, after Wave 2 survey data are available for the full study sample. The final evaluation 
report will be completed in summer 2005. During intervening periods, the study team will 
prepare a limited number of special-focus reports that address particular questions of interest 
to Congress or the U.S. Department of Health and Human Services. 
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Chapter I 



Background 



I n 1996, Congress authorized $50 million annually for five years in funding to states 
for programs that teach abstinence from sexual activity outside of marriage as the 
expected standard for school-age children. This funding was established through a 
new formula grant program created under Title V, Section 510 of the Social Security 
Act, authorized under the Personal Responsibility and Work Opportunity Reconciliation Act 
(PRWORA) of 1996. The funds became available to states in 1998 and are administered by 
the Maternal and Child Health Bureau. Currently, Title V Section 510 abstinence education 
is in its last funding cycle, and deliberations regarding reauthorization will begin shortly. 

Four years into the Section 510 abstinence education funding, the percentage of teens 
reporting that they have had sex has decreased, continuing a decline that started in 1991. At 
this time, however, no definitive research has linked the abstinence education legislation with 
these downward trends. Most people acknowledge that “abstinence works.” It is certain to 
prevent unwanted pregnancy, sexually transmitted diseases (STDs), abortions, and out-of- 
wedlock childbearing. However, an important question is: To what extent are abstinence 
education programs effective in persuading youth to be sexually abstinent and in changing 
teen sexual behavior? 

Empirical evidence on the effectiveness of abstinence education is limited. Moreover, 
most studies of abstinence education programs have methodological flaws that prevent them 
from generating reliable estimates of program impacts. Even the features of abstinence 
programs implemented, the curricula used, and the experiences of program staff and 
program participants are not well documented in a readily accessible way. To address this 
gap in information. Congress authorized a federally funded, independent evaluation of 
Section 510 abstinence education programs in the Balanced Budget Act of 1997 (Public Law 
105-33). Through extensive implementation, process, and impact analyses, the evaluation 
will strengthen the research base and knowledge about strategies for promoting sexual 
abstinence among youth and the benefits of various approaches to abstinence education. 
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This report presents interim findings from the congressionally authorized evaluation. 
The first in a series of reports from the evaluation, this report draws most heavily on four 
years of implementation experiences in a selected group of abstinence education programs 
funded under Tide V Secdon 510. This report also uses information from federal program 
monitoring reports, efforts by state and local evaluators, and policy and issue statements by 
various constituent groups and policy organizations. Later reports from the evaluation will 
present estimates of short- and longer-term program impacts, as well as studies on special 
topical areas. 

What Is the Context for the Title V Section 510 Abstinence Education 
Program? 

Despite a steady decline in the teen birthrate between 1991 and the present — from a 
high in 1991 of 62 births per 1,000 females age 15 to 19, to 49 such births in 2000 — 
concerns about teen sexual activity persist; 



■ In 1999, half of all high school students and nearly two-thirds of graduating 
seniors reported having had sexual intercourse (Centers for Disease Control 
and Prevention 2000). 

■ In 1999, one in five high school seniors reported having had sex with four or 
more partners (Centers for Disease Control and Prevention 2000). 

■ Nearly 480,000 babies were bom to teens in 2000, and 79 percent of these 
births were out of wedlock (National Center for Health Statistics 2002). 



The consequences of teenage sexual activity and out-of-wedlock childbearing are many 
and serious for teens, their families, their communities, and society. Over three-fifths of 
teen mothers live in poverty at the time of their child’s birth, and over four-fifths eventually 
live below poverty (Maynard 1996). Children bom to teen mothers often fare badly during 
infancy, early childhood, and their adolescent and adult lives. Compared with children born 
to mothers who delay childbearing until age 21 or older, children of teen mothers are more 
likely to grow up in homes that are not emotionally supportive or cognitively stimulating, to 
suffer from abuse and neglect, to repeat a grade in school, and to drop out of high school 
(Moore et al. 1997; Goerge and Lee 1997; and Haveman et al. 1997). 

In addition to its social and economic consequences, teen sexual activity also brings 
increased risks of STDs. In fact, teenage females have the highest rates of STDs of any age 
group. In the United States, more than 65 million people have an STD, and most are 
incurable viral infections (National Institute of Allergy and Infectious Diseases 2000). STDs 
may cause such lifelong complications as infertility, ectopic pregnancies, rniscarriages, 
stillbirths, intrauterine growth retardation, and perinatal infections. One STD, human 
papillomavims, is the primary cause of cervical cancer. Moreover, because of limitations in 
study design, the scientific evidence on the effectiveness of condoms in preventing STDs is 
inconclusive (National Institute of Allergy and Infectious Diseases 2000). 
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What Is the Title V Section 510 Abstinence Education Program? 

In this context of high rates of teen sexual activity and their serious public health and 
socioeconomic consequences, interest in abstinence education has increased over the past 
decade. As a result, in 1998 the federal government provided $50 million annually for five 
years for block grants to states to support abstinence education programs. States must 
provide $3 in matching funds for every $4 in federal funds, which results in a total of up to 
$87.5 million available annually for such programs. 

These abstinence education grants are allotted to states through a formula based on the 
proportion of low-income children in the state relative to the total number of low-income 
children for all the states. States then decide which programs to fund and at what level. 
Most states have disbursed their funding to numerous local agencies and organizations. 
However, a few states, such as Massachusetts, have retained their entire funding allocation 
for a single statewide initiative, such as a media campaign. 

Although abstinence education programs have been around for decades, the new 
investment raised the profile of programs that teach an unambiguous abstinence message to 
youth. The main factor that distinguishes the Section 510 abstinence education funding 
from the previous generation of federally funded abstinence education programs is the “A-H 
definition” (Title V Section 510 (b)(2) (A-H) of the Social Security Act), which specifies that 
an abstinence education program funded under the block grant must: 



A Have as its exclusive purpose teaching the social, psychological, and health gains to be 
realized by abstaining from sexual activity 

B Teach abstinence from sexual activity outside marriage as the expected standard for all 
school-age children 

C Teach that abstinence from sexual activity is the only certain way to avoid out-of- 
wedlock pregnancy, sexually transmitted diseases, and other associated health 
problems 

D Teach that a mutually faithful, monogamous relationship in the context of marriage is 
the expected standard of sexual activity 

E Teach that sexual activity outside the context of marriage is likely to have harmful 
psychological and physical effects 

F Teach that bearing children out-of-wedlock is likely to have harmful consequences for 
the child, the child's parents, and society 

G Teach young people how to reject sexual advances and how alcohol and drug use 
increases vulnerability to sexual advances 

H Teach the importance of attaining self-sufficiency before engaging in sexual activity 
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The Maternal and Child Health Bureau guidelines for these abstinence education 
programs offer the following interpretation of the A-H definition: “It is not necessary to 
place equal emphasis on each element of the definition. However, a project may not be 
inconsistent with any aspect of the abstinence education definition” (Maternal and Child 
Health Bureau 1997; Haskins and Bevan 1997). Providing instruction in or promoting the 
use of birth control would be inconsistent with the A-H definition. 

The A-H definition of abstinence education has generated controversy. Some 
abstinence educators and policymakers are critical of Section 510 abstinence education 
programs that do not emphasize all elements of the A-H definition. These criticisms have 
been especially strong for some preexisting health and teenage risk reduction programs that 
are perceived to have simply added limited abstinence education modules to their services 
specifically to gain access to Section 510 funding. On the other hand, some policymakers 
and health educators object to the Section 510 abstinence education programs primarily 
because of their restrictive definition of abstinence education. 

Still, since the inception of the Section 510 abstinence education funding, the number of 
abstinence education providers has increased dramatically. The funding has stimulated 
considerable discussion at the state and local level on the problem of teen sexual activity and 
raised local awareness and consideration of the role of abstinence education in local 
programs and policies. Supporters of abstinence education contend that such programs are 
effective because they are consistent with the developmental needs of adolescents for clear, 
consistent messages and boundaries. Their concern with sex education programs that teach 
about sexuality, contraceptives, and abstinence is that they send the mbced message that (1) 
teens should be abstinent, but (2), if they are going to have sex, they need information about 
and access to contraceptives. 

On the other side, proponents of comprehensive sex education programs contend that, 
while abstinence is preferred, broader sex education is essential because most teens are 
sexually active by the time they finish high school. Comprehensive sex educators contend 
that teens need to be taught about and provided access to contraceptives to reduce their risk 
of pregnancies and STDs. 

Despite these different perspectives on how best to address high rates of teen sexual 
activity, a common thread in the ongoing policy debate is an underlying interest in learning 
about effective program strategies that help youth make good choices that avoid risk-taking 
behaviors and promote healthy future lives. The evaluation of Section 510 abstinence 
education programs is designed to contribute much-needed knowledge on approaches to, 
and the effectiveness of, selected abstinence education programs. 
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What Is the Evaluation of Abstinence Education Programs Funded 
Under Title V Section 510? 

In fall 1998, the Assistant Secretary for Planning and Evaluation, U.S. Department of 
Health and Human Services, awarded a competitive contract to Mathematica Policy 
Research to conduct an independent evaluation of Section 510 abstinence education 
programs. The evaluation addresses three important questions: 



■ What are the nature and underlying theories of the abstinence education 
programs supported with Section 510 funding? 

■ What are the implementation and operational experiences of local 
communities and schools that have received Section 510 abstinence 
education funding? 

■ What are the impacts of abstinence education programs? 

- How successful are they in changing the knowledge, attitudes, and 
intentions of youth? 

- How successful are they in reducing teen sexual activity among youth? 

- How do they change the risk of pregnancy and STDs? 



To address these questions, the evaluation includes an extensive implementation and 
process analysis, focused on 11 abstinence education programs, as well as rigorously 
designed impact studies of 5 of these programs. The implementation and process analysis 
uses program documents; program observations: focus groups with program participants, 
parents, and other area youth; and interviews with program staff and community leaders to 
document and understand important features of the range of programs that have been 
implemented. The impact study uses longitudinal survey data for groups of youth randomly 
assigned to the abstinence program in the community or to a control group. 

The evaluation design was developed and implemented with guidance from a technical 
workgroup composed of individuals with demonstrated expertise in the myriad critical 
aspects of this complex research agenda (see Appendix A). In addition, the evaluation team 
held meetings with numerous interest groups to solicit their input regarding the evaluation 
questions, site selection criteria, and data collection strategies. 
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Site Selection 



The first step of the evaluation entailed selecting abstinence programs for study. The 
evaluation team first called and met with numerous state officials and experts across the 
country to identify promising programs for inclusion in the evaluation. Grant applications 
and program documents then provided additional detail on program goals, target population, 
activities, size, and curricula. The evaluation team visited and observed 28 abstinence 
education programs across the nation. After extensive communication with abstinence 
experts and DHHS staff, 11 programs were invited and agreed to participate in the 
evaluation (Table 1). Although not a representative set of Section 510 abstinence education 
programs, these 11 programs are judged to offer a rich range of program strategies and 
implementation settings for study. 

Five of the programs included in the evaluation are referred to as “targeted” programs, 
as they target their services to specific, identifiable groups of youth. In addition to providing 
valuable information about program implementation, each of these programs offers the 
potential to provide rigorous evidence on the effectiveness of abstinence education program 
strategies in reducing teen sexual activity and other risk behavior. The other six programs 
represent a range of multifaceted, community-wide initiatives that aim to alter youth 
behavior through stimulating systemic change. These programs are using their abstinence 
education funding in a variety of ways to increase public awareness of the problems of teen 
sexual activity, to change community norms and attitudes, to engage parents and encourage 
stronger parent-child communications, and to engage youth in abstinence education and 
support services. Including these community-wide initiatives in the evaluation adds breadth 
to an understanding of strategies for changing youth behavior. 

For several reasons, though, rigorous impact studies of these community-wide 
abstinence program initiatives are not possible. First, these programs often use their 
abstinence funding to form or become part of a larger network of services for teens, making 
it impossible to separate the effects of the abstinence program from those of other programs 
or providers. Second, the target population often is not easily identifiable, since, by intent, 
these programs aim to change the norms and behavior of an entire community. Finally, 
systemic change is a long-range goal, and the time frame for the impact analysis and federally 
funded evaluation is too short to allow such change to be measured. 

In selecting programs for the evaluation, the goal was to maximize the overall knowledge 
that would be generated, including providing operational lessons and impact estimates based 
on a range of program approaches and implementation settings. Moreover, all 1 1 programs 
met two additional criteria: (1) each conformed to, and in some cases was based on, a 
theoretical framework that links its services to changes in youth knowledge, attitudes, 
intentions, and behavior: and (2) each appeared to be reasonably well implemented, which 
included having its core services operational, committed staff in place, and key partnerships 
established. 
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Table 1: Abstinence Education Programs 
Participating in the Evaluation 



Location and 
Sponsoring Agency 


Principal Program Components 


Target 

Population 


Targeted Programs (Impact, Implementation, and Process Analysis) 


FL (Miami) 

Youth service agency 


Elective class offered daUy, all year to girls in middle schools. Urban 
setting; diverse student population. Curriculum: ReCapturing the Vision 
and Vessels of Honor 


Grades 6-8 


MS (Clarksdale) 
Community health 
agency 


Mandatory weekly year-long curriculum. Rural community; extremely 
poor population. Curriculum: Revised Postponing Sexual Involvement and 
Sex Can Wait 


Grades 5-6 


SC (Edgefield) 

Youth service agency 


Five-session mandatory curriculum with voluntary enrollment in 
weekly or biweekly character clubs. Middle-income community. 
Curriculum: Heritage Keepers 


Grades 6-9 


VA (Powhatan) 
County health 
department 


36-session mandatory curriculum. Middle-income community. 
Curriculum: Reasonable Reasons to Wait; The Art of Loving WeK'and 
Choosing the Best 


Grades 8 and 10, 
with 9th and 11th 
grade boosters 


WI (Milwaukee) 
Social service agency 


Voluntary after-school program; two hours daily all year for multiple 
years. Seven-week summer program. Poor, inner-city community. 
Curriculum: Families United to Prevent Teen Pregnancy 


Grades 3-8 


Community-Wide Initiatives (Implementation and Process Analysis) 


lA (Cedar Rapids) 
Not-for-profit coalition 


Abstinence curriculum; community resource library; classroom 
presentations: workshops for parents and educators; Baby Think It 
Over dolls; speakers; mentoring program; teen panels 


Emphasis on 
middle school 
youth 


NY (Monroe County) 
County health 
department 


Not Me Not Nowvr\&d\di campaign; abstinence curriculum; parent 
outreach program; interactive web site; mentoring program 


Emphasis on 9-14 
year olds 


SC (statewide) 

Youth service agency 


Abstinence curriculum; character clubs; school assemblies; 
training/information for medical providers, faith workers, parents, and 
media personnel; numerous collaborations and partnerships 


Middle and high 
school youth 


TX (Fort Bend County) 

Community-based 

organization 


Abstinence curriculum; separate youth development programs for girls 
and boys; peer education program; school assemblies; community 
training; parent education programs; parent resource center; 
community events; medical provider involvement 


9-17 year olds, 
with a heavy focus 
on middle school 
youth 


TX (McLennan County) 

Community-based 

organization 


Abstinence curriculum; school assemblies; character education in 
elementary schools; mentors; media spots; medical provider training; 
faith-based partners; resource library 


Emphasis on 10- 
14 year olds 


UT (Tooele County) 
County health 
department 


Abstinence curriculum; parenting class; self-esteem days for 5th-8th 
graders; self-esteem classes for high-risk youth; Baby Think It Over 
dolls; peer educators; school fairs; billboards and newsletters; 
merchant involvement; faith-based linkages 


9-14 year olds 
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For the five programs targeting services on particular groups of youth, three additional 
site selection criteria were applied in order to ensure the feasibility of conducting a rigorous 
impact analysis of each program: 



■ The program’s services and activities differ from what participants 
otherwise would receive. The evaluation contrasts outcomes of youth 
receiving program services with those of control group youth who do not. If 
participants would receive similar services without the program, the 
evaluation would essentially compare a program to itself, leading to negligible 
estimated impacts. 

■ The program can readily adapt to evaluation procedures. It was 
important that a program be able and willing to adapt to critical evaluation 
procedures without adversely affecting the basic program services. It was 
essential that the evaluation be able to address programmatic concerns while 
not compromising the evaluation’s credibility. 

■ Both the programs and the target population need to be large enough 
to support the sample size requirements for a rigorous study. It was 
important that a program be able and willing to adapt to critical evaluation 
procedures without adversely affecting the basic program services. It was 
essential that the evaluation be able to address programmatic concerns while 
not compromising the evaluation’s credibility. 



While the abstinence education programs selected are considered interesting and well- 
implemented programs, they are not necessarily better than or representative of the more 
than 700 abstinence education programs funded under Section 510 and operating 
nationwide. Many of the other Section 510 abstinence education programs are being 
examined in evaluations that states and localities have funded themselves, and these other 
studies will provide rich and important detail on the range of abstinence strategies and their 
effects (Mathematica Policy Research, Inc. 2001). 



Implementation and Process Evaluation 

The implementation and process analysis documents the experience of organizations 
and communities applying for and receiving abstinence education funding in both the 
targeted and community-wide program sites. It describes the abstinence interventions 
implemented — the program goals, the underlying theoretical framework, and the specific 
curriculum elements covered by the program. It examines the target population and 
community context, and reports the participants’ experiences with the program. It also 
details the organizational structure of the abstinence education program models, identifying 
those models that have been used and the factors associated with successful implementation. 
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The implementation and process analysis uses three primary data sources: (1) review of 
program documents and records; (2) interviews and focus groups with program staff, school 
staff, community leaders, parents, and program pardcipants; and (3) on-site program 
observations. Program documents and records provide important background information 
on the program’s objectives and message, as well as data on youth served. Most programs 
selected for the evaluation have promotional materials that are distributed to youth and, in 
some cases, to the community at large. Some have web sites or use the media to deliver the 
abstinence message and to identify community resources available for youth. Program 
documents also often describe staff qualifications and background, present staff training 
materials, and include written communications among abstinence providers. Program 
records provide valuable information on youth served, resource requirements, and costs. 

Interviews with program and school staff also convey important information on 
program goals and implementation, as well as on the more intangible aspects of commitment 
to the program message and the importance of helping youth. Interviews and focus groups 
with program participants provide insights into the experiences of youth, their perceptions 
of the program, and the intangible factors that lead to program success. Focus groups with 
parents yield additional insights on the needs of youth and how programs can best engage 
parents in helping their children make good choices. Finally, firsthand program observations 
are invaluable for assessing how the program messages and services are delivered and 
received. 

The data collection efforts to support the implementation and process analysis were 
extensive. Evaluation team members made multiple rounds of site visits to all programs 
included in the evaluation, as well as to many others ultimately not included in it. During 
these site visits, the evaluators conducted executive interviews with numerous staff 
members, facilitated the focus groups, and observed program delivery. In addition, phone 
calls and ongoing review of program records provided further detail on program 
implementation. Finally, an extensive communications effort and outreach to constituent 
groups and policy organizations yielded rich information on the context of abstinence 
funding and the types of programs supported. 



Impact Evaluation 

To date, the evaluation team has successfully implemented a scientifically rigorous 
impact study design in the five targeted program sites. Study enrollment is completed, and 
longitudinal tracking of youth is ongoing. Because of the importance of having large sample 
sizes and sufficient followup of program and control youth, no impact estimates are available 
for inclusion in this report. This report does, however, provide detail on the key features of 
the study design, the characteristics of the youth served by these programs, and plans for the 
impact analysis and reporting. 
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As discussed in greater detail later, critical features of the impact analysis study design 
are: 



■ The Impact Study Uses Experimental Designs in AU Sites. In each site, 
program effectiveness will be measured by comparing outcomes of eligible 
youth who were randomly assigned to the program or to a control group. 

■ Sample Sizes Are Between 400 and 700 Youth per Site. Large sample 
sizes protect against the possibility of failing to detect true program impacts 
simply because the study lacks statistical power. To achieve adequate sample 
sizes, youth were enrolled in the study from fall 1999 through fall 2001. 

■ Followup of the Sample Will Continue for Up to Three Years After 
Study Enrollment. The first wave of data collection occurred at “baseline” 
as participants were first enrolled in the program or the control group (fall 

1999 through fall 2001), the second wave occurs 6 to 12 months later (fall 

2000 through spring 2002), and the third wave will occur 18 to 36 months 
after initial sample enrollment (spring 2002 through fall 2003). 

■ Data Collection Procedures Respect the Rights of Students and 
Parents and Protect the Privacy of Respondents. The evaluation uses an 
active parental consent process whereby the parent or guardian must provide 
written consent before a student is eligible to participate. The evaluation also 
uses confidentiality protections designed to ensure that no one from local 
schools — including teachers, administrators, and program staff — has access 
to the student surveys. 

■ Student Surveys and Data Collection Procedures Are Designed to 
Maximize the Accuracy and Reliability of Student Responses. Research 
indicates that youth may underreport socially undesirable behaviors, such as 
sexual activity and other risk-taking activity. Furthermore, abstinence means 
different things to different people. The surveys for the study measure 
specific, clearly defined behaviors. Students provide their answers on self- 
administered forms under the supervision of trained, independent 
interviewers. 

The first report on short-term program impacts will be released early in 2003, once the 
second wave of survey data is available for the full study sample. The final impact analysis 
report will be released in summer 2005. 
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Chapter II 



Early Implementation Experiences 
OF Abstinence Education 
Programs 



T he first four years of Title V Section 510 funding for abstinence education have 
generated a wealth of experience on how local communities and schools have 
designed and implemented abstinence programs and how youth have responded 
to them. Interviews with program staff, parents, and students, as well as 
observations of what and how the abstinence message is conveyed, have yielded rich 
information on program operations, program models, youth response, and local agency 
partnership opportunities and challenges. This chapter discusses the experiences of 
communities, schools, and youth with abstinence education programs, focusing primarily on 
the 11 programs selected for the evaluation. The analysis also draws on observations of the 
uses of abstinence education funding more generally. Federal monitoring efforts document 
the funding allocations and the types of programs supported nationwide. State and local 
evaluations provide additional detail on how the state grants are disbursed to local 
communities and the range of programs and activities supported. 

The experience of these early years of funding has produced five important conclusions 
that can guide future fiscal and programmatic decisions: 

1. Section 510 abstinence funds are changing the local landscape of 
approaches to teenage pregnancy prevention and youth risk avoidance. 

Despite an initial debate in some states over whether and how to spend 
abstinence education funds, aU states applied for funding at some point and are 
using the monies in innovative ways to promote abstinence from sexual activity 
as the healthiest choice for youth. 
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2. Most abstinence education programs offer more than a single message of 
abstinence. Examples of curricula and program components from sites 
participating in the evaluation indicate the diverse, creative, and often complex 
nature of many initiatives. 

3. Most participants report favorable feelings about their program 
experience. Youth respond especially positively to staff who show strong and 
unambiguous commitment to the program message. They also seem to like 
programs that deliver an intensive set of youth development services to enhance 
and support the unambiguous abstinence message. 

4. Abstinence programs face real challenges addressing peer pressure and 
the communication gulf between parents and children. Sexual activity often 
elicits only casual mention among youth, and is tolerated and even promoted by 
their peer culture. Many programs attempt to address peer pressure, in part, 
through parent involvement. Yet, for many programs, engaging parents has 
proven to be extremely challenging. 

5. Local schools are valuable program partners, but establishing these 
partnerships is sometimes difficult. Their broad access to youth makes 
schools logical and important partners for many programs, but some schools 
resist collaboration with abstinence education programs. Sometimes schools 
resist because of competing priorities; at other times, resistance stems from 
debate about health and sex education policies. 



Title V Section 510 Abstinence Funds Are Changing the Local Landscape 
OF Approaches TO Preventing Teen Pregnancy 

Funding for abstinence education has contributed to the evolving national struggle to 
address the social and economic consequences of teenage sexual activity, teenage 
childbearing, and out-of-wedlock births. Following considerable and sometimes rancorous 
controversy over whether and how to spend the $50 million in annual abstinence education 
block grant funds, every state ultimately applied for the money. In each of the successive 
funding years, nearly all of the states and territories took advantage of the funds available to 
them. 

The decision of states to avail themselves of the abstinence education funding has 
stimulated considerable discussion at the state and local level about the role of abstinence 
education. One consequence of the discussions has been a dramatic increase in the number 
of school districts that report teaching abstinence. In 1988, only 2 percent of school districts 
in the United States reported teaching abstinence as the sole way to prevent pregnancy and 
STDs; by 1999, this figure had risen to 23 percent of districts (Darroch et al. 2000). Indeed, 
in many communities, abstinence education programs have become an important 
component of the network of programs serving teens and their families. 
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Three factors help explain the current extent of acceptance of abstinence education. 
First, the Section 510 abstinence education programs implemented across the country have 
gained support because they are more than ‘‘J^st Say No” programs. They offer a breadth 
of services and activities designed to support youth, equip them with knowledge and 
decision-making skiUs to help them make good choices, and provide them with constructive 
activities that are fun and widely perceived as good for kids. Second, many programs focus 
on middle school students, where there is general agreement about the appropriateness of a 
strong abstinence approach to sexual education. Finally, the coalitions formed at the local 
level to deliver the abstinence message often bridge a gap in ideological perspectives, 
allowing abstinence education programs to coexist with other programs that respond in a 
variety of ways to the needs of teens and their communities. 

The Section 510 abstinence education funding supports more than 700 programs 
nationwide and has resulted in a tremendous range of new programmatic approaches to 
preventing teen sexual activity and out-of-wedlock pregnancy (Maternal and Child Health 
Bureau 2000). The amounts awarded to local grantees by states vary widely. For example, 
among the programs included in the evaluation, annual award amounts range from $50,000 
to over $800,000. States have awarded abstinence education grants to community-based 
organizations, local school districts, local health departments, faith-based organizations, and 
universities, among others. The funding guidelines encourage states to fund grantees’ efforts 
directed at local priority needs, 
and the diversity of uses of 
funding reflects this intent 
(Maternal and Child Health 
Bureau 1997). Local grantees use 
funds for community-based 
projects, as well as for evaluation 
and program monitoring, 
technical assistance and training, 
media campaigns, advisory 
councils, resource and 
communication networks, toll- 
free hotlines, and satellite 
conferences. 



Number 
of States 


Grantees 


38 


Community-based organizations 


29 


Local Boards of Education/School Districts/Schools 


27 


Youth Serving Organizations 


23 


Local Health Departments 


22 


Faith-Based Organizations 


19 


Universities 


17 


Local Coalitions /Partners hips/ Advocacy Groups 


17 


Consultants/Contractors 


16 


Media/Research Firms 


15 


Health Care Organizations 


14 


Non-Profit Organizations 


SOURCE: Maternal and Child Health Bureau (2000). 



The core of aU these efforts is a message about the benefits of abstinence from sexual 
activity, which most often is delivered through a curriculum-based program in a school 
setting. This approach, which often has a youth development component, ffequendy is 
referred to as character-based education or “assets building.” Other common efforts include 
adult mentoring, peer mentoring, parent education, before- and after-school programs, and 
recreational-based activities. More broadly based initiatives include curriculum development, 
public awareness campaigns, and community partnership development (Maternal and Child 
Health Bureau 2000). 
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The majority of programs aim their abstinence message at middle school students. 
However, some target a wider age spectrum, starting younger and persisting longer. Many 
also target high school youth, and a few target out-of-school youth. Resource constraints 
lead many programs to limit their selection of a target population. 

Focusing on youth of middle school age or younger has helped some communities 
resolve the debate between those who favor an ‘‘abstinence-only” approach and those who 
favor an “abstinence-plus-contraception” approach. The emerging consensus that the 
middle school years are an appropriate time to offer these interventions suggests some 
agreement that a message of abstinence is an important foundation for all efforts at youth 
risk avoidance and pregnancy prevention. 



Abstinence Education Programs Offer More than a Single Message 
OF Abstinence 

A common perception is that abstinence education programs focus narrowly on teaching 
youth the single message of abstaining from sexual activity before marriage. Observations of 
the programs participating in the evaluation suggest that many of the programs include 
multiple components designed to reinforce and support their abstinence message. For 
example, the program curricula used by the five targeted programs address a broad range of 
issues, from building self-esteem to understanding and aspiring to healthy marriages and 
parenthood, and to teaching skills that will help youth make — and follow through on — 
good decisions (Table 2). 



Table 2 : Curriculum Topics of Abstinence Education Programs 
Participating in the Impact Evaluation 





FL 


MS 


SC 


VA 


WI 


Building Self-Esteem 


V 


V 


V 




V 


Developing Values/Character Traits 


V 


V 


V 


V 


V 


Formulating Goals 


< 


< 


< 


< 


V 


Making Decisions 




< 


< 




< 


Avoiding Risky Behanor 


< 








V 


Maximizing Communication 


< 


< 


< 


< 


V 


Strengthening Relationships 


< 


< 


< 


< 


V 


Understanding Development and Anatomy 


< 


< 


< 


< 


< 


Understanding STDs 












Withstanding Social and Peer Pressure 


< 


< 


< 


< 


< 


Addressing Consequences /Self-Control 


< 


< 


< 


< 




Resolving Sexual Conflicts 


< 


< 


< 


< 




Learning Etiquette and Manners 












Aspiring to Marriage 


< 








< 


Understanding Parenthood 








< 





Source: Program curricula manuals. 



The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report 



25 



15 



In addition to the abstinence education curricula, program services illustrate the breadth 
of activities offered to youth. Weekend summits, community roundtable discussions, 
lending libraries and websites, essay contests, door prizes at school dances, ''abstinence 
coupon books’’ for local businesses, summer programs, family retreats, and program 
recognition ceremonies are examples of the range of activities offered to program 
participants. 

In general, the program curricula, activities, and opportunities provided to youth reflect, 
either implicidy or explicidy, various underlying theories of adolescent behavior and the 
implied logic models that explain the knowledge, atdtudes, in tendons, and behavior of 
youth. The most influendal theories of adolescent behavior incorporate muldple factors 
believed to shape youth behavior, including: 



■ Youth background and personality characteristics. Youth background 
characterisdcs include demographic factors, biological factors such as age and 
physical development, religion, and school and community characterisdcs. 
Important personality characterisdcs include impulsivity, risk-taking 
proclivides, sense of efficacy, and youth temperament, which also can 
influence youth atdtudes and behavior (Costa et al. 1995; and Jessor and 
Jessor 1977). 

■ Family attitudes and relationships. This includes parental atdtudes, 
values, and communicadon, as well as the influence of siblings. The quality 
of reladonships and extent of interacdon with parents are related to the 
degree of engagement in risky behaviors (Feldman and Brown 1993; Blum et 
al. 1987; and Whitbeck et al. 1993). Siblings, too, influence youth behaviors, 
including the number and birth order (Rodgers and Rowe 1988). 

■ Youth attitudes, values, and knowledge. Atdtudes and values strongly 
influence adolescent behavior. Knowledge about risks of certain behavior 
affect the decision-making and behavior of adolescents. Youth who perceive 
their vulnerability as high, the consequences as serious, and the costs greater 
than the benefits are most likely to avoid risk (Bandura 1977 and 1986; and 
Rosenstock 1988). 

■ Peer relationships and social influences. Atdtudes and values of peers 
are powerful predictors of adolescent atdtudes and behavior (Evans 1976; 
McGuire 1964; Schinke et al. 1985; and Fishbein and Ajzen 1980). 
Avoidance of risky behavior requires that youth understand social pressures 
and that they have the skiUs to resist those that are negadve. Interacdons of 
individuals and their environment influence youth behavior (Hawkins and 
Catalano 1992; and Klitzner 1993). Negadve atdtudes, behaviors, and 
interacdons can be risk factors, while posidve bonds formed with a school, 
peer group, or community can be protecdve factors. 
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With varying emphasis, abstinence education programs recognize these social, 
developmental, and community antecedents and mediators of youth behavior and draw on 
one or more of four complementary strategies for promoting abstinence and other healthy 
behavior: (1) Helping youth learn skills to deal effectively with social influences and peer 
pressure; (2) providing them with information to better assess the benefits and costs of their 
actions; (3) altering family and community norms and supports; and (4) promoting healthy 
development through age-appropriate, healthy-activity choices. 

Social Influences and Peer Pressures. All 11 abstinence education programs in the 
evaluation address social and peer pressures. Lessons on decision-making and 
communication, and active -learning exercises (such as role-playing) often are used to help 
youth develop and apply critical skills needed in situations that involve peer pressure. 
Discussions of attitudes, beliefs, and values help youth distingmsh themselves from 
perceived peer norms. The Teens in Control program in Clarksdale, Mississippi, for example, 
uses videos to depict teens in relevant situations and then engages program youth in role- 
playing exercises so that they can apply decision-making and communication skills. 

Benefits Assessment. Many abstinence education programs seek to reduce the 
motivation to engage in risky behaviors by teaching youth to recognize the consequences of 
such behaviors and the benefits of avoiding them. They use various strategies to alter 
motivation, including exercises to build confidence, self-esteem, problem-solving abilities, 
and conflict negotiation skills. 

The Not Me Not Now program in Monroe County, New York, has as its cornerstone a 
media campaign that makes adolescents, parents, and the community more aware of the 
consequences of teenage sexual activity and stresses positive future options for teens to 
motivate them to remain abstinent. The media campaign includes paid television and radio 
advertising, billboards, 5,000 posters in schools, mouse pads for public school students ages 
9 to 14, t-shirts, educational materials for parents and schools, and a quarterly newsletter 
mailed to youth ages 9 to 14. Parents are targeted through workshops, as well as through a 
widely distributed pamphlet and video. The advertisements convey the program's message 
by drawing on local youth to act in the commercials. Local youth also serve on an advisory 
panel to help shape the media messages and gauge response. 

Several of the programs work to alter benefits assessments through teaching about the 
values in and what constitutes a good marriage. The very heavy emphasis on the institution 
of marriage in some of the programs reflects a belief that the lack of understanding of, or 
role models for, marriage results in its being undervalued. This undervaluing of marriage is 
believed to contribute to casual and early sexual relationships. The ReCapturing the Vision 
program in Miami, Florida, is an example of a program that attempts to change knowledge 
about and the perceived value of marriage. The topic of marriage is covered over a period 
of more than a month of daily classes in which program participants paint their own small 
“hope chests,’’ discuss extensively what makes for a good partner in Ufe, “plan” for their 
own weddings, and hold a mock wedding at a local hotel. The selected “bride” draws on the 
lessons on relationships and partner qualities in selecting the “groom.” Parents of the bride 
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and groom, as well as program participants from all schools, attend the mock “wedding,” 
which culrriinates in vows of chastity until a real wedding. 

Family and Community Norms and Support. Particularly the community-wide 
programs and the more intensive targeted programs often attempt to enhance youth’s 
involvement with their families, peer groups, schools, and community through offering a 
multifaceted set of services, activities, and educational and training opportunities. They may 
be designed to mobilize broad, interrelated factors within the larger community to 
strengthen positive influences on individual behavior. 

The Families United to Prevent Teen Pregnancy (FUPTP) program in Milwaukee, Wisconsin, 
and the ReCapturing the Vision program in Miami, Florida, for example, both work hard to 
improve parent-child communication and to enhance participants’ involvement with their 
families through means such as monthly parent meetings, a weekend family retreat, and 
home visits. The ReCapturing the Vision program seeks to develop positive peer relationships 
by running a class-appointed “court system” for students who cause problems and an annual 
Teen Talk Symposium in which teen and celebrity panelists address questions on 
relationships and sexual issues. 

Both FUPTP and the ReCapturing the Vision programs aim to strengthen participants’ 
commitment to school through a heavy emphasis on school performance, with report card 
checks and dedicated homework/ tutoring time. Both programs are intensive; they meet 
daily throughout the school year and offer program participants the opportunity to enroll for 
more than one year. They address skills needed to support community engagement, with 
opportunities for community service and lessons on social etiquette through dining at local 
restaurants. These programs aim to provide youth with a value system that will help them 
develop their decision-making skills, communication sltiUs and relationships, and goal 
setting. In addition to participants’ attitudes and values, they also focus on self-esteem. For 
example, one strategy used by ReCapturing the Vision, an aU-girls program, is to provide 
participants with “makeovers” to improve their self-image. 

Developmental Needs. Many programs offer age -appropriate activities and supports 
designed to fill unmet psychological and emotional needs, develop psychosocial competence, 
and ease teenagers’ transition to positive, independent, and productive adulthood. For both 
the ReCapturing the Vision and the 
FUPTP programs, a primary vehicle for 
engaging youth is giving them a strong 
sense of identity with a group that 
embraces positive values, such as 
community, responsibility, leadership, 
trust, and respect for others. This 
group identity is achieved in a number 

program, a local business pays to have 
suits designed and tailor-made for each 



of ways. In the ReCapturing the Vision 



FUPTP “PLEDGE” 

I, of the Families United to Prevent Teen Pregnancy 
Program, promise not to become a teen parent. 

I will abide by the rules of Rosalie Manor Incorporated, 
my parents, teachers, and community in which I live. 

I will not become a part of illegal drugs, drug abuse, 
crime, or gang-related activities. 

I will forever carry myself as a future leader and 
illustrate FUPTP pride. 



The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report 



28 



program participant. Both programs help foster a sense of identity in the public’s eyes 
through highly publicized public rallies to support the choice of abstinence. In the FUPTP 
program, participants write their own ‘haps” that reflect what they’ve learned from the 
program and each day recite a program pledge. 



Most Participants Feel Favorably About Their Program Experience 

Youth tend to respond especially positively to programs when the staff are 
unambiguously committed to abstinence until marriage and when the program incorporates 
the broader goal of youth development. Young teachers who are public about their own 
commitment to abstinence appear to be very successful in engaging program youth. The 
Heritage Keepers Program in South Carolina, for example, uses teachers who demonstrate this 
unqualified endorsement of abstinence until marriage. The program trains these teachers to 
be direct and to communicate their commitment to abstinence. Observations during site 
visits suggest that committed and outspoken teachers are effective in capturing the attention 
of students and getting them to listen and question. 

Most programs have limited resources and so must make trade-offs between the 
intensity and duration of services they provide each participant and the overall number of 
youth they serve. As observed in classrooms and reported during focus groups, youth seem 
to respond especially favorably to the intensive programs because they are tailored to the 
developmental needs of youth and provide services and activities that go far beyond the 
classroom curricula. These programs often include field trips, weekend activities, end-of- 
the-year celebrations, and local and national motivational speakers, all of which are geared to 
helping youth make informed choices about their behaviors. During focus groups, students 
in one program reported that they are learning about goals; values; high and low self-esteem; 
high- and low-risk behaviors; good and bad consequences; responsibility; social skills; and 
abstinence from sex, dmgs, and alcohol— and learning not to become a teen parent. 

Not all programs have met with enthusiasm, however. The less intensive programs, in 
particular, more often fail to engage students fuUy and encounter dissatisfaction among 
youth with program services. Participants in one such program complained that the class 
was boring and was “just another class” that “didn’t offer much benefit.” Students in 
another site acknowledged that some students make fun of the program’s slogan. 



Challenges Addressing Peer Pressure and the Communication Gulf 
Between Parents and Adolescents 

Abstinence programs face real challenges in addressing peer pressure and the 
communication gulf between parents and adolescents. Testimony from youth about their 
perceptions of what is going on among their peers reflects, among other things, the extent to 
which they have been inundated with media messages, images, and thoughts about sexual 
activity at a very young age. Youth are observing, thinking about, and using sexual activity as 
a system for peer classification. 



The Evaluation of Abstinence Education Programs Funded Under Titie V Section 510: Interim Report 



Student Reports of Peer Sexual Activity 



19 



I think there are three different groups ... in our school. We have divided into the popular people, the kind- 
of-popular people, and the not-popular people, and the kind-of-popular people are like maybe they’ll give a 
peck on the cheek, but then the popular people are already like touching. 

It [depends] on the person . . . because there’s some people who, you know, our age now maybe don’t want 
to be so fast, but it is some fast people our age who, you know, do whatever, whenever, however. 

Most people I know, if they’re . . . being pressured, they’ll just do it. They won’t — jJ don’t know anybody 
who would say, like, ‘‘You know what? I can’t do this,” or ‘'You know what? We need to talk about this.” 

My school, it’s like they hang out a lot outside of schools. Our relationships tend to be the kids, not the 
seventh grade, but the eighth grade they are really, really close and they go past kissing a lot. . . . Either you’re 
in the don’t-do-it, you- want- to-do-it, or you-are-doing-it crowd, and a lot of people fall into that are-doing-it 
crowd, and those would be the popular kids in our school. 

They’ll like go home on the bus, they live in the same neighborhood, and they know their parents aren’t 
coming home to like six. They come over, one of them goes over to their house, and she said they have oral 
sex . . . most of the time it’s just oral sex. It’s not like hardcore, real sex. 

SOURCE: Focus groups held in Rochester, New York, for the Not Me Not Now program, conducted by 

Harris Interactive, Inc. Sessions were held separately with boys and girls, and included youth in 
grades 5 and 6, and in grades 7 and 8, from a range of urban and suburban schools. 



Constructive activities, particularly during after-school hours, can be an antidote to peer 
pressure, but such activities are not always available to youth. During focus group sessions, 
youth acknowledged that sexual activity takes place during unsupervised hours after school, 
as well as at large parties and on “dates” or in small gatherings of friends on weekends. 
When asked about what they usually do after school, many said they are bored. Many go 
home and watch television, talk on the phone, do homework, or baby-sit siblings. In one 
community, parents and their children both said, ‘We just don’t have anything here,” 
indicating few options or places for youth to go after school. In another community, there is 
a youth recreation center, but not within walking distance of the school. 

Good communication between parents and adolescents can also counter peer pressure 
(Miller 2001; and Blum and Reinhart 2001). Recognizing this, many programs try to bridge 
the gap in parent-child communication in any of three ways. First, they often try to help 
youth feel more comfortable discussing with their parents issues related to sex. Second, they 
may try to engage parents actively in the programs, inviting them to program events with 
their children. Third, many have special parent-focused components that aim to strengthen 
the parents’ ability to interact more effectively with their children. 

Both the Abstinence Education Initiative (AEI) Coaiition of Equipping Youth in Cedar Rapids, 
Iowa, and the Not Me Not Now program in Monroe County, New York, report increases in 
parent-child communication as a result of homework assignments requiring parent 
participation. Testimony from parents during focus groups confirms that programs are 
experiencing some success at improving parent-child communication. In focus groups, 
parents report that youth are becoming more comfortable talking about sensitive topics. 



The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report 





COPY 




30 



20 



“asking questions that they didn't ask before" and “opening up conversations." One parent 
commented that she is “embarrassed to talk about some of these things, but [her son] talks 
and makes [her] more comfortable." Another parent said, “My son has calmed down a lot; 
we talk to each other more." Particularly in the intensive programs, parents generally agree 
that the program is having a positive effect on their children by giving them some important 
skills, or “building blocks," with which to have positive interactions and communication 
with others, including themselves. 

Program efforts to involve parents in special program events with their children often 
succeed in bringing parents to the events. For example, parents attend weekend rallies held 
by the FUPTP program in Milwaukee, Wisconsin, by Heritage ComiDUlUty Services program in 
South Carolina, and by ReCapturing the Vision in Miami, Florida. 

On the other hand, promoting and sustaining active involvement in parent education 
and enrichment programs has been difficult. Despite widespread parent enthusiasm for 
programs, getting more than a small fraction actively involved has proven to be a major 
challenge for virtually all programs. In the Not Me Not Now program in Monroe County, 
New York, and the Youth Abstinence Education Program in Tooele, Utah, workshops on parent- 
child communication have been widely advertised, but attendance has not measured up to 
the extensive outreach campaign. Free pamphlets and videos have been made available to 
parents in local supermarkets, but these too have not been taken at the rate expected. The 
Not Me Not Now program is now considering ways to work within existing parent groups, 
such as those convened by local churches or local adult education programs. 

Even when a program includes a focus on the whole family, engaging parents can be a 
struggle. In FUPTP, staff members often register students for the program through a home 
visit, during which they explain the importance of parental involvement. The program holds 
monthly meetings for parents for which they provide food, transportation, and child care. 
They send home a regular newsletter and provide parents with progress reports. The 
curriculum uses take-home handouts for parents. Despite these efforts, parent involvement 
remains low. 



Partnerships with Local Schools Are Valuable but Often Difficult 
TO Establish 

Local schools are usually important partners in abstinence education programs. Schools 
provide unparalleled access to youth as a captive audience. Other than through the media, 
there is really no other way to reach so many youth on a regular basis. Furthermore, 
although the media can reach extraordinary numbers of youth, they do not have the targeted 
focus on youth development that many educators feel is critical to decisions regarding sexual 
attitudes and behaviors. However, establishing partnerships with schools is sometimes 
difficult. 
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Based on observations and reported experiences of the programs visited during site 
selection and more in-depth examination of the 11 sites that are the focus of the evaluation 
effort, it appears that schools generally become partners in abstinence educadon funding 
through one of four models of organizadonal structure. Three involve cooperation between 
the grantee (the local agency that received Section 510 funding from the state) ^and others 
(Figure 1): 



■ The one-to-one program model refers to an organi 2 ational structure in 
which a community-based agency is awarded a grant to provide abstinence 
education to youth in a school-based setting, and establishes a partnership 
with the local schools or school district. The Powhatan County Health 
Department in Powhatan, Virginia, for example, is providing an abstinence 
education curriculum to students in the county’s middle and high schools. 
Program staff, hired by the health department, have a cooperative agreement 
with the school district to teach these classes. 



Figure 1: Models of Organization Among Section 510 
Abstinence Education Programs 



ONE-TO-ONE MODEL 





COALITION MODEL 




The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report 



32 



BEST COPY AVAIIABILE 



22 



■ The wheel program model depicts an organizational structure in which the 
Section 510 grant recipient spearheads an abstinence education initiative in 
several different settings, often drawing on other community resources such 
as schools, local businesses, health care providers, or other social service 
organizations to assist with newly developed efforts. The Fort Bend Abstinence 
Leadership and Resources for Teens initiative in Fort Bend, Texas, as well as the 
McLennan County Collaborative Abstinence Project in McLennan County, Texas, 
have spearheaded numerous and diverse abstinence initiatives in their 
communities. In both cases, these new initiatives include involvement of the 
medical community and development of a local information, and resource 
center. The Youth Abstinence Education Program of the Tooele, Utah, County 
Health Department has launched school, extracurricular, parent, and 
community awareness activities and programs. Examples are a “self-esteem 
day” for fifth through eighth graders, games and door prizes at program- 
sponsored or supported parties and dances, parent-teen conferences, and 
abstinence messages contained in discount coupons for local stores. 

■ The coalition program model is an organizational structure in which 
multiple participating organizations contribute staff to a newly formed entity 
and joindy oversee program initiatives and the Section 510 funding. For 
example, coalition members in the AEI Coalition of Equipping Youth 
program in Cedar Rapids, Iowa, include a county school district, a corps of 
high school students who present skits in other schools, a volunteer 
organization operated by physicians’ spouses, a group that focuses on 
substance abuse, and a program staffed by parenting teens. Funding is 
coordinated through the coalition, and monthly meetings of coalition 
members provide an opportunity for members to coordinate and build on 
one another’s efforts. 



The fourth model, though not used by any of the programs selected for the evaluation, is 
a single-agency model. For example, a school that receives funds to redesign its own sex 
education curriculum may operate independendy rather than in a cooperative relationship 
with any other organization. 

Regardless of the organizational structure, creating and sustaining partnerships between 
abstinence education programs and local schools often requires enormous persistence and 
resources. AU the programs in the evaluation have established such partnerships, but with 
varying degrees of challenge and success. Some programs have been welcomed arid given 
extensive support. Some have had to pursue partnership agreements on numerous fronts 
over long periods of time, conducting community discussions, seminars with parents, and 
repeated conversations with principals, district superintendents, and local school boards. 
Still others have received litde ongoing support or have been denied access and forced to 
seek partnerships in other districts or with other organizations. 
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Where districts and schools are reluctant to support abstinence education programs, it is 
often because of conflicts over the sex education curriculum. In an effort to appear even- 
handed, the Cedar Rapids, Iowa, school district excluded from its classrooms both the 
abstinence educators and educators from Planned Parenthood. Still, the AEI Coalition of 
Equipping Youth program steadily expanded its access to county schools as the positive 
reputation of its school-based program in one school district spread. The public schools in 
Waco, Texas, rejected the abstinence education funds. As a result, the McLennan County 
Abstinence Education Project focuses its school-based services in surrounding school districts 
and works with faith-based and other community service organizations to reach high-risk 
youth in the city. And in South Carolina, the Heritage Keepers program was rejected in some 
communities that did not want abstinence education as the sex education curriculum in their 
schools. 

Lack of support from school staff is often a factor jeopardizing a school-based 
abstinence education program. Based on the reported experiences of leaders of the 11 
programs in the evaluation, as well as a number of other abstinence education programs 
visited during site selection, skepticism can emanate from the principal or from classroom 
teachers. Even when principals invite a program into a school, unless they work to 
underscore its merit, classroom teachers may withhold support. Teachers’ resistance to a 
program’s mission or unwillingness to coordinate with program staff, as many sites have 
reported, can undermine the effectiveness of program operations. 

Waning support can result from the emergence of new priorities. For example, in 
Powhatan, Virginia, an increase in school violence usurped much of the principal’s time and 
resources, so that the abstinence program received less attention. The current national 
emphasis on school accountability for student achievement is increasing the priority given to 
“core” rather than “non-core” courses, and abstinence education programs often struggle in 
this environment. 

Lack of support from a principal also can emerge as a result of staff turnover. As in 
several schools participating in the evaluation, the principal who invited the program into the 
school leaves, and the new principal’s agenda and priorities do not include strong support 
for the abstinence education program or the organization that runs it. The Teens in Controi 
program in Mississippi, for example, worked very hard for an entire year to gain the full 
cooperation and welcome from two of the three districts in which it planned to operate. In 
one of these districts, a state takeover resulted in a new principal being hired to “turn the 
school around.” As a result, the new principal had extremely limited opportunity or 
incentive to give any priority to the abstinence program. 

Lack of dedicated space for the abstinence program can be a symptom of weak school 
support. Unless an abstinence program is replacing an existing school offering, space 
availability is often an issue. Several of the programs involved in the evaluation face space 
constraints and often get shuffled around. This instability in physical location can further 
undermine support for the program, even among program participants. Uncertainty 
regarding program location and the inability to establish a secure “home” (either to leave 
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materials /resources or to create a physical identity for the program) can create frustration 
and ultimately jeopardi 2 e the interest and commitment of students and teachers. 

Programs can strengthen their partnerships with schools through visibility. They can do 
this by making sure that the principal and the teachers understand the program’s mission and 
curriculum, and that they are regularly informed on issues related to the program and its 
participants. For programs with time and flexibility, providing a direct link to the needs of 
classroom teachers — such as through the provision of time for homework or tutoring 
assistance, as is done in the FUPTP program in Milwaukee — can help integrate the program 
into the school’s existing agenda. Finally, most schools will welcome programs that achieve 
visibility through popularity with participants and parents; those that make a real investment 
in youth will be rewarded. An example is the success of the grassroots efforts of parents 
from the Iowa College Community School District in convincing the school board to adopt 
the abstinence education program offered through the AE I Coalition of Equipping Youth. 

Among the partners of programs participating in the evaluation, some principals and 
schools have been deeply committed to the abstinence education initiative. In such cases, 
positive, mutually beneficial relationships for all — the school, the program, the participants, 
and the parents — have generally emerged. 



More Lessons to Come 

The collective state and local abstinence education program experiences from across the 
country now constitute a sizable body of information to inform the dialogue over 
approaches to reducing teen sexual activity. The level of attention now focused on teen 
sexual attitudes, behaviors, and consequences should help determine how best to assist 
communities in selecting programs to meet their local needs. This attention can, as noted in 
the Charleston Post and Courier create “a healthy dialogue among teachers, clergy, parents, 
health professionals and students to share experiences from each perspective and come to a 
consensus of what’s best for the children” (Lawrence 2001). 

Getting the most out of these experiences depends critically on learning what impacts 
various approaches to abstinence education have for the youth they serve. The findings 
from the impact evaluation component of this evaluation will be critical to shaping the 
future policies and programs to best meet the needs and interests of youth. 
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Chapter III 

The Foundation for Assessing 
THE Impacts of Abstinence 
Education Programs 



I n 1997, Congress authorized, and its staff requested, a scientifically rigorous impact 
evaluation of the abstinence education programs funded under Tide V Section 510 of 
the Social Security Act. Policymakers, school officials, community leaders, program 
staff, and parents all want to know the extent to which particular program strategies 
succeed. They want to know for whom these strategies work and to understand the 
ingredients of that success. They also want to gather informadon that will guide program 
improvement for any groups identified as not responding well to pardcular strategies. 

The early operadonal findings discussed in Chapter II provide a cridcal foundadon for 
subsequent reports to address these quesdons of program effectiveness. Much has been 
learned about school and community responses to the absdnence educadon funding, the 
range and nature of coalidons formed, the design and operadonal experiences of the 
programs, and the responses of youth and their parents. Evidence on the impacts of the 
absdnence education programs, however, is not currendy available, because obtaining 
definitive and rigorous evidence on program impacts is a complicated, long-term process. 

Over the past four years, the evaluation effort has laid the foundadon for a careful, 
comprehensive, and rigorous assessment of program impacts. The research team has 
selected five targeted programs for the impact evaluadon, built the partnerships needed to 
conduct the evaluadon, enrolled samples large enough to support reliable esdmates of 
program impacts of each program, and designed and implemented a rich and muldpronged 
data collection strategy to support the evaluadon of each program. The impact evaluadon 
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will build on this foundation to determine the extent to which the abstinence programs in 
the evaluation achieve six specific goals: 

1. Strengthen knowledge and attitudes supportive of abstinence 

2. Induce more youth to embrace abstinence as a personal goal 

3. Reduce sexual activity among youth 

4. Persuade sexually experienced youth to become or remain abstinent 

5. Lower the risk of STDs 

6. Lower the risk of nonmarital pregnancies 



Evidence on attainment of these goals is being developed through a scientifically 
rigorous impact evaluation design, careful and comprehensive data collection, and detailed 
and deliberate analysis and reporting. The impact evaluation design avoids the limitations of 
most prior research on abstinence education programs. Few previous studies, for example, 
used rigorous experimental research designs to generate program and control groups. Those 
that did use experimental designs usually randomized entire classrooms or schools rather 
than individual students, which severely reduced their effective sample sizes.* Few were able 
to use independent professional data collectors. Finally, few were able to track outcomes of 
their sample members over an extended follow-up period. Consequently, results usually 
pertain to outcomes of youth before they reached the age when many were engaging in 
sexual activity. 

Scientific Rigor in the Study Design 

The scientific rigor of the impact study design rests on four key elements. It begins with 
the selection of strong, well-implemented, replicable program models. Second, the impact 
study uses a rigorous experimental design to create program and control groups within each 
site. Third, the sample enrollment period was long enough to generate adequate sample 
sizes to support reliable impact estimates. Finally, the impact evaluation includes a 
reasonable length follow-up period to ensure that relevant changes in behavioral outcomes 
can be measured. 

The impact evaluation examines five programmatic strategies geared to the 
needs of the local communities (Table 3). Measuring impacts for a range of program 
models promotes the goal of identifying and documenting abstinence education strategies 
appropriate to varied local needs and contexts. For example, the Florida and Wisconsin 



'When classrooms or schools are the unit of randomization, the “effective sample size” is substantially 
lower than would be if students were the unit of randomization. This is because of the high within-class or 
within-school correlation (Kish 1965). 
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programs serve mainly youth from single-parent households; these programs are intensive 
and include strong components on relationship development and maintenance, as well as 
understanding and appreciation for the institution of marriage. In the Mississippi program 
site, many youth live in large, multigenerational households isolated from the broader 
community. The program in this community is delivered through the schools and 
emphasizes both basic Imowledge development and components focused on managing peer 
pressure. Youth in the South Carolina and Virginia programs live in communities that 
mirror “middle America.” The program in Virginia is a low-cost, school-based intervention, 
while the one in South Carolina is a more comprehensive and intensive youth development 
initiative. These choices of program strategies reflect community characteristics and 
perceptions of how best to serve youth, given local needs and the resources and constraints 
of the partner schools. 

One implication of the variation in program interventions and services is that it is not 
possible to reach a single judgment about the efficacy of abstinence education. Such a 
judgment would only be possible if there were a single, well-defined intervention, one that 
could vary in its “dosage” across sites but is similar in nature across all sites. In the case of 
the Section 510 abstinence education programs, however, the interventions and services vary 
considerably across program sites and sometimes even within a program site. In the absence 
of definitive evidence on the efficacy of a specific abstinence education approach, this 
variation is a natural result of the funding opportunities available through Title V Section 
510. In addition, the variation in the abstinence education programs provides the 
opportunity to learn about the effectiveness of different programmatic strategies. 



Table 3: Program Interventions and Services Received by the Control Groups 



Program Location 


Program Intervention 


Control Group Services 


FL (Miami) 


Elective class offered daily, all year to girls in middle schools 
(ReCapturing the Vision and Vessels ofHonoj) 


Other elective class 


MS (Clarksdale) 


Mandatory weekly year-long abstinence education curriculum 
(Revised Postponing Sexual Involvement and Sex Can Wail} 


Regular health class 


SC (Edgefield) 


Five-session mandatory curriculum with voluntary enrollment 
in weekly or biweekly character clubs (Heritage Keeper^ 


Five-session mandatory abstinence 
curriculum without character clubs 


VA (Powhatan) 


36-session mandatory curriculum (Reasonable Reasons to Wait; 
The Art of Loving Weti; and Choosing the Besi) 


Regular health class 


WI (Milwaukee) 


Voluntary after-school program; two hours daily all year for 
multiple years (Families United to Prevent Teen Pregnant^ 


Regular after school programs; no 
special services 



The impact evaluation uses an experimental design. In an experimental design 
study, program slots are filled by youth who are selected at random from a larger pool of 
eligible and appropriate youth (Figure 2). Random assignment procedures divide youth into 
a program group that has access to the abstinence education program and a control group 
that does not receive the program, but may receive regular or alternative services. The 
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contrast in services being studied varies depending both on the nature and intensity of the 
program services and the experiences of the control group (see Table 3). 

Longitudinal tracking of both the program and control group youth begins at the time of 
sample enrollment and continues for 18 to 36 months, depending on the time of initial 
enrollment. The comparison of outcomes for these two groups over time provides the basis 
for judging impacts of the program. 

The experimental design offers the best means of measuring, with a known degree of 
certainty, how successful the programs are overall and how well they serve key subgroups of 
youth within a site. This is because, with careful implementation, the only systematic 
difference between the program and control youth should be their access to the program. 
As a result of the random assignment, the program and control groups have similar 
demographic and background characteristics within any study site (Figure 3) and they are 
exposed to a common school and community context. 
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However, the characteristics of sample youth vary across study sites due to a 
combination of factors, including program targeting practices and differences in the program 
and community characteristics. For example, the average age of youth at the time of sample 
enrollment ranges between 10 in the Wisconsin program site, which delivers its services 
through an after-school program, to 13 in the Virginia program site, which serves exclusively 
eighth graders. The proportion of sample youth who are nonHispanic black ranges from a 
low of 12 percent in ^e Virginia program site to over 80 percent in two other programs, one 
of which operates in a rural southern community, the other in an inner-city setting. The 
proportion living in two-parent families ranges from 37 percent to more than 75 percent. 



Figure 3: Demographic and Background Characteristics Are Similar for Program and 
Control Youth Within Each Site 



Average Age NonHispanic. Black 




FL MS SC VA WI FL MS SC VA WI 



Two-Parent Family 




□ Program Group ^ Control Group [ 



Source: Wave 1 surveys administered by Mathematica Policy Research, Inc., to sample youth at or near enrollment. 
Note: None of the differences between program and control group youth is statistically significant at the 10 percent 

level in a two-tailed test. 



The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report 




30 



Random assignment generates, in each study site, program and control groups consisting 
of youth who, on average, are subject to similar family rules and express similar attitudes and 
values about abstinence before the program group is exposed to abstinence education 
services (Figure 4). For example, the proportion of youth who say their parents have strict 
rules about companions they spend time with varies across sites between 15 and 45 percent, 
but is similar for program and control youth within each site. Between 62 and 83 percent of 
sample youth in each study site reported believing that “having sex as an unmarried teen 
would make it harder to subsequently have a good marriage,” and between 16 and 35 
percent hold the view that “having sex is a way to tell someone you love them.” In all cases, 
however, the views of program and control youth are nearly identical within each site. 



Figure 4: Family Rules and Attitudes about Teen Sex Are Similar for Program and 
Control Youth at Srunple Enrollment 
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Source: Wave 1 surveys administered by Mathematica Policy Research. Inc., to sample youth at or near enrollment 

Note: None of the diri'erences between program and control ^up youth is statistically significant at the 10 percent level in a 

two-tailed test 
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A major advantage of the random assignment design is that it protects against selection 
bias in the impact estimates for the individual programs studied. Other evaluation designs 
are vulnerable to selection bias, which can seriously undermine the credibility of their results. 
Some evaluations, for example, have relied on comparisons of outcomes for participants in 
“elective” programs and youth at the same site who, for some reason, do not participate. 
Others compare outcomes for program youth with youth who responded to local or national 
surveys. In both cases, there is a strong possibility that the participants differ in some 
preexisting but unobservable way from the comparison group. These preexisting differences 
may lead to biased estimates of program impacts. 

Pre-post comparison designs have other defects. Comparisons of measures for 
participant groups before and after their involvement in a program can be affected not only 
by the program but also by natural maturation effects. For example, data from the National 
Longitudinal Survey of Adolescent Health show that the percentage of teens who have ever 
had sex increases from 9.6 percent at age 13 to 19.6 percent at age 14. Thus, using a pre- 
post design to measure program impacts on abstinence would seriously bias the results 
toward estimates of no impacts or possibly even adverse impacts. 

Studies that rely on comparison samples drawn from existing survey databases can be 
weakened by both bias and unreliability. Some studies, for example, compare program 
participants with respondents to the Youth Risk Behavior Survey or the National 
Longitudinal Survey of Adolescent Health. Such study designs have the added 
complications arising from noncomparability of survey instruments, data collection methods, 
and timing of the data collection (Santelli et al. 2000). 

Carefully designed and implemented experimental design studies can both overcome 
these weaknesses and offer unanticipated bonuses for programs and policymakers. When 
program resources are not sufficient to serve everyone, many youth will not receive the 
abstinence education program services, regardless of whether there is an experimental-design 
evaluation or not. Random assignment is often fairer than commonly used practices such as 
“first come, first served” or referral systems to allocate scarce program resources. Random 
assignment designs also can provide valuable information about the magnitude of “unmet” 
demand for the program services. Assuming that the evaluation design is implemented so 
that programs operate at capacity, the size of the control group provides a lower-bound 
estimate of unmet demand. At the same time, the operational experience with outreach and 
recruitment provides qualitative information regarding how thorough and successful the 
outreach efforts are and may provide tips on how to strengthen future outreach efforts. 

One limitation of a random assignment design for measuring program impacts arises if 
any of the programs has major spillover effects. If, for example, youth who are assigned to 
the program group interact with youth in the control group in ways that transfer the benefits 
of the program intervention to peers in the control group, the random assignment study 
design will underestimate program impacts. Similarly, if the presence of an intervention in 
the school or community significandy alters the overall school or community climate in 
important ways, this could lead to underestimates of program impacts. The overall 
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judgment of the evaluation team is that, for each of the five sites included in the impact 
evaluation, spillover effects are expected to be very small in relation to the direct effects on 
those who participate in the program. Nonetheless, this is an issue that has received ongoing 
attention by the evaluation team and that is addressed in the follow-up surveys with 
students.^ 

The impact evaluation has large sample sizes of between 400 and 700 youth per 
site. Large sample sizes protect against the possibility of failing to detect true program 
impacts simply because the study lacks statistical power. It is important that, if no 
statistically significant program-related impacts are detected on sexual activity or on risks of 
STDs or pregnancy, for example, one of two conditions holds: (1) there really was no 
impact of the program at all, or (2) any program impact was sufficiently small as to be of no 
importance to policymakers or practitioners. 

What constitutes a sample size large enough to detect true impacts depends in large part 
on the nature of the program. Generally, low-intensity or short programs have smaller 
impacts and, thus, require larger sample sizes to ensure that true impacts are picked up in the 
andysis. The opposite is generally true of programs that are longer or more intensive. 

The originally planned one-year period of sample enrollment for the evaluation was 
extended to three years in order to generate samples large enough to ensure detecting 
meaningful program effects and to avoid false claims of no effects. Final sample sizes per 
site are expected to vary between 443 (280 program/163 control) and 700 (371 program/329 
control) students. Table 4 presents estimates of changes in outcomes the study will be able 
to detect using reasonable standards of statistical power and precision, given these sample 
sizes and given national estimates of the prevalence for selected outcomes. For example, the 
study will be able to detect true program impacts on the percentage of students who are 
sexually experienced of 7.2 percentage points or larger in the site with 700 youth in the study 
sample and of 11.2 percentage points or larger in the site with 443 youth in the sample. 

To guard against errors that might arise based on findings from small sample sizes with 
low statistical power, no impact evaluation results will be released until data for the full study 
sample are available. Results based on just the first one or two years of sample enrollment 
would run a risk of missing true impacts simply because of small sample sizes. 

The study sample is being followed for up to 36 months. The data collection 
schedule balances the need to release study findings at the earliest point possible with the 
importance of ensuring that study findings offer reliable guidance for policy and practice 
decisions. Two waves of follow-up surveys are planned. The wave 2 follow-up survey is 



^This is an issue that was of sufficient concern during the study design that an external review of the study 
design was commissioned to ensure that there was strong professional support for the random assignment 
design adopted for the study. 
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Table 4: Minimum Detectable Changes in Outcomes 



Minimum Change Detectable‘s 





Estimated Prevalence 


Largest 


Smallest 


Outcome Measure (Wave 3) 


of Outcome® 


Sample 


Sample 


Taken Virginity Pledge 


14.9% 


±6.0% 


±9.3 


Sexually Experienced 


24.1% 


±7.2% 


±11.2 


Abstinent at Follow-up*^ 


86.5% 


±5.8% 


±8.9 


At Risk of Pregnancy‘s 


17.3% 


±6.4% 


±9.8 


Sample Size 




700 


443 


Program Group 




371 


280 


Control Group 




329 


163 



* These estimates are based on computations from the National Longitudinal Survey of Adolescent Health data. 
National prevalence estimates for youth at different ages have been weighted by the age distribution of the Title 
V Section 510 abstinence education program evaluation sample in the construction of these estimates. 

^ Minimum detectable differences are calculated based on the actual sample sizes, adjusted for anticipated 
nonresponse to follow-up surveys. A 95 percent confidence interval and an 80 percent power requirement 
were used. 

Defined as never had sexual intercourse or not sexually active in past 90 days, 
d Defined as sexually experienced and did not use a highly effective method of contraception at last intercourse. 



being administered 6 to 12 months after initial study enrollment (when the wave 1 baseline 
survey was administered), and the wave 3 follow-up survey will be administered between 18 
and 36 months after enrollment. The interval between sample enrollment and the wave 3 
survey depends on the age of youth at enrollment and the latest calendar date when surveys 
can be administered given the reporting schedule. Under this plan, it is possible to analyze 
both short-term impacts on knowledge, attitudes, and intentions of youth related to 
abstinence and longer-term impacts on behavior. 

Because so few youth engage in sexual activity before entering high school, outcome 
estimates based on wave 2 outcome data from middle-school years would miss program 
impacts on behaviors that most often would emerge at later ages. Indeed, a shortcoming of 
previous abstinence education evaluations has been a follow-up period that does not extend 
beyond the middle school years. Nationally, only 12 percent of males and 8 percent of 
females under age 13 have ever had sex (tabulations of ^e National Longitudinal Survey of 
Adolescent Health). It is important to have the data collection period extend as long as 
possible in order to measure behavioral outcomes at ages where the prevalence of the 
behavior is high enough that changes in behavior will be observed. 

The follow-up period for this evaluation is such that almost two-thirds of the study 
sample will be 14 to 18 years of age by the time of wave 3 followup and no youth will be 
younger than age 12. Even with the extended follow-up period, however, only six percent of 
the study sample will have reached ages 18 and 19, when over half their peers are expected 
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to be sexually active. To address the potential need for even longer followup, the data 
collection procedures and plans for the evaluation are designed to accommodate longer 
followup, if resources were to become available. 

Careful and Thorough Data Collection Plans and Procedures 

Plans and procedures for the data collection in the impact evaluation are designed to 
capture the high-quality data needed for a thorough evaluation. A conceptual framework for 
the program intervention strategies, which is consistent with the main theories of adolescent 
behavior discussed earlier, dictates the data collected and the timing of those data (Figme 5). 

This framework acknowledges that the decisions youth make regarding sexual activity 
and other risk-taking behavior (Column IV) depend critically on a range of antecedent 
factors (Column I), including demographic and background characteristics of the youth, 
characteristics of their parents and their families, and the school and community context in 
which they have been raised. For all youth, these antecedent factors are mediated by current 
parental attitudes, values, and supports: the attitudes, knowledge and relationships of the 
youth: and the current school and community context in which youth live (Column III). 



Figure 5: Conceptual Framework for Evaluating Abstinence Education Programs 



I. ANTECEDENTS OF k 


II. EXTERNAL MEDIATING k 


III. NATURAL MEDIATING y 


IV. KEY BEHAVIORAL L 


TEENAGE SEXUAL 1 


FACTORS AFFECTING 1 


FACTORS 1 


OUTCOMES 1 


ACTIVITY I 


BEHAVIOR 1 


1 


1 
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There are two means by which the abstinence education programs (or any other 
intervention) operate to potentially alter the key outcomes of interest. One is by direcdy 
altering youth behavior. The other is through affecting the natural mediating factors, for 
example, by providing parents with knowledge and tools to better guide their children in 
sound decisionmaking: by changing the attitudes, knowledge and relationships of youth in 
ways that reduce their inclination to engage in risk-taking behaviors: or by changing the 
school and community climate in ways that are more expectant and supportive of 
abstinence. 

The first wave of student surveys administered near the time of enrollment in the 
evaluation study gathers information on the antecedents of teen sexual activity and baseline 
values of the natural mediating factors (Columns I and III). Wave 2 and wave 3 surveys 
gather information to mark changes in the natural mediating factors and the key outcomes 
(Columns III and IV). 

A number of critical issues relate to the design and administration of these surveys to 
support the rigor of the impact study. These include: 



■ Protecting the privacy of sample members 

■ Using questions that will generate valid and reliable measures of the 
constructs of interest 

■ Dealing with normal reluctance to report sensitive and socially undesirable 
information 

■ Addressing the fact that youth may have different definitions of what 
“abstinence” means 

■ Ensuring that questions are age appropriate 

■ Avoiding contamination of the programs’ abstinence message by the data 
collection itself 



The rights and privacy of sample members and their parents are paramount. 
Only youth whose parents have given active parental consent for their child to participate in 
the study are included in the study sample. Moreover, youth themselves must actively 
consent to each wave of data collection. The privacy of student responses is protected 
through a rigorous system that relies on professional, independent data collectors: that 
permits no personal identifying information on any survey form or data file containing 
survey responses: that maintains secure data files: and that has the protection of a Federal 
Certificate of Confidentiality (HRSA-00-15). 

Survey questions were selected with attention to issues of the validity and 
reliability of the core constructs for the evaluation. Each question included in any of the 
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three surveys has been mapped to one of the core constructs in the conceptual framework 
(Figure 5 above). Moreover, in determining the particular questions that would be asked to 
address each construct, careful attention was paid to the experience of prior studies with 
similar populations, including the validity and reliability of measures for different target 
populations and when questions were administered through different data collection modes. 
For example, questions about school and family draw heavily on the National Longitudinal 
Survey of Youth and the National Education Longitudinal Study of 1988; questions on 
youth attitudes about sexual activity draw heavily on questions used in prior studies of 
abstinence education programs, such as Values and Choices (Olsen et al. 1991), Teen Aid 
(Weed et al. 1998), Responsible Social Values Program (Adamek 1993), Best Friends (Best 
Friends Foundation 1997), and Sex Respect (Weed and Olsen, no date); questions about 
other risk-taking behaviors, such as drinking and using drugs, draw heavily on questions in 
the Youth Risk Behavior Surveillance Survey (Centers for Disease Control and Prevention 
1993) and the National Longitudinal Survey of Youth (Card 1993); and questions about 
romantic relationships and actual sexual experiences draw on the National Longitudinal 
Survey of Adolescent Health (Udry and Bearman 1998), the National Survey of Family 
Growth (Card 1993), and the Youth Risk Behavior Surveillance Survey (Centers for Disease 
Control and Prevention 1993). 

Each of the survey questionnaires was pretested with small groups of youth. After 
revisions, they were then reviewed by key staff in the five programs participating in the 
impact evaluation, by the Federal Office of Management and Budget, and by the University 
of Pennsylvania’s Institutional Review Board. In addition, staff from various constituent 
groups and policy organizations reviewed the survey questionnaires, provided useful insights, 
and made helpful suggestions. 

Youth may not want to report sensitive and socially undesirable information. 
Some respondents may feel uncomfortable reporting accurate information on questions 
about sexual intercourse and may distort their responses in the direction that they perceive as 
socially desirable. Moreover, the problem of underreporting behavior that is considered 
socially undesirable may be exacerbated for youth who participate in abstinence programs, 
given the strong and unequivocal message of these programs. 

To minimize the underreporting of sensitive behaviors, as well as to protect the privacy 
of the study sample, the evaluation uses self-administered survey data collection, maintains 
the strictest standards of confidentiality, and informs the survey respondents about them.^ 
The data collection procedures ensure that no one from the local schools — including 
teachers, administrators, and counselors — has access to students’ survey responses. School 
and program staff are not allowed to participate in the data collection: trained interviewers 
conduct all survey data collection and focus groups. As soon as the student surveys are 
completed, the interviewers immediately separate student contact information from the 
surveys and remove them from the school grounds. 



methodological experiment was conducted to assess whether using personal data-recording devices 
increased reporting of sensitive behaviors. It did not have any such effect for the evaluation sample. 
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Before the students complete the surveys, the interviewers assure all respondents that 
their answers will be kept confidential and will not be shared with anyone. The consent 
forms sent home to parents, as well as the assent forms given to students, make it clear that 
no individual-level data from the surveys will be reported. Rather, information on individual 
students will be combined into groups for analysis and reporting purposes. 



Survey administration methods protect student privacy 

■ Most students complete the surveys by themselves in the presence of trained interviewers who 
can answer questions about the survey administration. Younger sample members and those with 
poor reading skills have the survey read to them, but they mark their own responses. 

■ Trained, professional interviewers employed by Mathematica Policy Research, Inc., conduct all 
survey data collection. 

■ All surveys are removed from the school premises immediately upon completion. 

■ No personal identifying information is included on the survey instruments. 

■ A Federal Certificate of Confidentiality protects the student data. 



Youth may have different definitions of abstinence. The primaiy goal of the Section 
510 abstinence education programs is to persuade youth to abstain from sexual activity. 
Thus, it is very important that survey questions accurately measure this outcome. 

Survey questions on abstinence from sex are difficult to design, since abstinence means 
different things to different people. Some consider abstinence to mean refraining from all 
intimacy except for kissing and holding hands, while others consider abstinence as anything 
except sexual intercourse. Participation in abstinence education programs may lead some 
youth to change their definitions of what constitutes sexual activity and abstinence. Failure 
to address such program-induced changes in definitions could result in a downward bias in 
the reporting of abstinence by program youth relative to control youth and thereby limit the 
detection of true program impacts. 

It is essential to ask in the clearest way possible about specific behaviors of greatest 
interest. To have reliable measures of sexud activity, the evaluation survey instruments 
measure whether study youth have ever had sexual intercourse. Since program and control 
youth are likely to have the same understanding, on average, of what sexual intercourse is, 
this measure has greater reliability than survey questions diat ask simply about abstinence 
from sexual activity. 

Outcome measures must be age appropriate. The survey and administration 
methods for the study are sensitive to the social and emotional development of sample 
youth. The abstinence education programs target youth in their preadolescent and 
adolescent years, and measurement of outcomes must reflect that age span. Some programs 
serve youth as young as third or fourth grade. Measures of program impacts for 
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preadolescent youth may be quite different from those for adolescent youth. For example, 
questions related to sexual intercourse are not appropriate for preadolescent youth, given the 
low prevalence of the behavior and, more importantly, the age inappropriateness. 



The evaluation survey instruments for youth below grade seven do not ask 
whether the respondent has had sexual intercourse. 



The survey was designed to avoid contamination of the abstinence message. The 
Section 510 abstinence education programs promote a strong message that teenagers should 
postpone sexual activity until marriage. The programs do not promote use of contraception, 
on the premise that such information is inconsistent with program goals and sends a mixed 
message to youth. 

However, a careful evaluation must measure the main outcomes of an abstinence 
education program. The evaluation must be able to measure whether program participants 
do or do not abstain from sex and whether program participants do or do not engage in 
behaviors that risk pregnancy and exposure to STDs. This requires that the survey questions 
about sex measure similar behaviors for the program and control youth and be detailed 
enough to measure exposure to risks of pregnancy and STDs. Moreover, it is critical that 
the study’s informed consent procedures are consistent with asking youth these sensitive 
questions. 

Accurate assessment of whether programs affect risk of STDs and pregnancy must take 
into account the behaviors of those youth who become sexually active. Among youth who 
are sexually active, exposure to unwanted pregnancy and STDs depends, .among other 
factors, on the use of condoms or other contraceptives. Therefore, the evaluation survey 
instruments ask a limited number of questions about use of condoms and other 
contraceptives. These questions are seen by and are asked only of youth who have already 
stated that they have had sex, and they are designed so that they do not provide information 
that the abstinence programs themselves avoid communicating. 

Future Analysis and Reporting Plans 

Over the three-year sample enrollment period, which ended in fall 2001, the evaluation 
team secured cooperation from 3,300 youth and their parents to participate in the impact 
evaluation. To date, 3,081 of these youth have completed the wave 1 survey. The wave 2 
survey has been administered to those youth enrolled during the first two study years, with 
1,791 completing this survey thus far. In spring 2002, the wave 2 survey will be administered 
to the remaining sample, and the wave 3 survey will be administered to those who enrolled 
in the study sample during the first year of sample enrollment. Sample youth will continue 
to be tracked through surveys and, in some cases, school records through fall 2003. 
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Furthermore, program operations and community context will be monitored continuously 
throughout the remainder of the study period to support the evaluation. 



Table 5: Interview Schedule and Sample Sizes, by Time of Sample Enrollment 



Sample Enrollment 


Total N 


Wave 1 


Wave 2 


Wave 3 


Fall ’99/Spring ’00 


1,040 


Fall ’99/Spring ’00 


Fall ’00 


Spring/Fall ’02 


Fall ’00 


901 


Fall ’00 


Spring ’01 


Fall ’03 


Fall ’01 


1,359 


Fall ’01 


Spring ’02 


Fall ’03 


Total Number 


3,300 


3,081 


2,970^ 


2,805^ 



® Estimated number of completed surveys. 



A report on the effects of the programs in achieving their short-term goals of changing 
knowledge, attitudes, and near-term behavioral choices will be completed in early 2003 once 
wave 2 survey data are available for the full study sample. The final study evaluation report 
will be completed in summer 2005. During intervening periods, the study team will prepare 
a limited number of special-focus reports that address particular questions of interest to 
Congress or the U.S. Department of Health and Human Services. 
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